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ABSTRACT 

Project CARES (Career Awareness Resources for 
Exceptional Students) of the New York City Public Schools served 67 
Spsmish- speaking students w.'th intensive social and emotional needs 
and limited English proficiency. The students were ages 10 to 21. The 
project provided students with instruction in English as a Second 
Language, Native Language Arts, and bilingual instruction in the 
content areas. The project sponsored a joint venture in which 
students, parenLs, supervisors, community residents, and local 
business entrepreneurs introduced students to the job market. 
Evaluation of the pr^'ject found that it raised the self-esteem and 
confidence of the students. It equipped staff to work together to 
achieve common goals and equipped students with instruction in how to 
do peer tutoring. The need for additional training for project staff 
is noted. This evaluation report covers staffing, implementation and 
outcomes by individual objectives, and a case history. An appenr»-!X 
describes data collection and analysis procedures. (JDD) 



jt* ****** ******* ********************* 



* Reproductions supplied by EDRS are the best that .-an be made 

* from the original document* 




C'\REER AWARENESS RESOURCES 
FOR EXCEPTIONAL STUDENTS 
(PROJECT CARES) 
1990-91 



FINAL EVALUATION PROFILE 



CAREER AWARENESS RESOURCES 
FOR EXCEPTIONAL STUDENTS 
(PROJECT CARES) 
1990-91 



FINAL EVALUATION PROFILE 



NEW YORK CITY BOARD OF EDUCATION 



H. Cari McCaU 
President 

Irei^ H. ImpcUizzeri 
Vice President 

Carol A. Gnesscr 
Wesiina L. Matthews 
Michael i. Petiides 
Luis 0. Reyes 
Ninfa Segana 
Members 

Joseph A, Fcmaixkz 
ChancelU^ 



DIVISION OF STRATEGIC PLANfNING/RESEARCH & DEVBLOPMENT 



Robin Winner 
Executive Director 



If is As prfky tl^N«w^ 

«Bd en^Ioyntait pobasi, as tfqtmi bf law. Inqoihei re|«iiaf oomdMaoB ippmpriaifl Uw* my tw tiirecud to 
MercedM A. Neifield. Dir9csr>r. OSkaof Equl Opporoiniiy, 110 Livni|itoe SirM. Btwikiyn. New York 11201; or lo 
Piwcttr. Office for Civil Rialiia. DepBimeator Educaoon. 26 Fedml Ptaa. Room 33.ia New York. New York 10278. 



7/76/91 



ACKNOWLEDGMENTS 



This report has been prepared the Bilingual, MuUicultural, and Early Chfldhood Evaluation 
Unit of the Office of Research, EvaluatioD, aod Assessment. Thanks are due to Cyril Wallace for 
collecting the data and writing the report. 

Additional copies of this report are available from: 

Dr. Tomi Deutsch Bemey 

Office of Research, Evaluation, and Assessment 

New York City Public Schools 

110 Livingston Street, Room 732 

Brooklyn, NY 11201 

(718) 935-3790 



FOREWORD 



The body of this report is preceded by an Extract which presents an overview of salient points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is foUowed by the body of the report, titled Pro gram Assessment . This includes such 
information as stafOng, program implementation, and out(X)me and in^lementation objectives. 
Instructional objectives are presented first, foUowed noninstructional objectives. The report then 
addresses those a^ts of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in Appendix A following the text. 
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Offioe of Research, Evaluatknt, and AiMUiKnt 
New York Qty PiMc Scbo(^ 

E.S.E.A. Title VII Evaluation Profile 

CAREER AWARENESS RESOURCES 
FOR EXCEPTIONAL STUDENTS 
(Project CARES) 

Grant Number: TOOSADOISS 
1990-91 
EXTRACT 

PROJECT DIRECTOR: Yvonne Tavares-Jimenez FUNDING CYCLE: Year 1 of 3 
SITES 



School Community School District Grade Levels Enrollment (Total 67) 

P.S-12X 11 6,7,8,9 38 

P.S.754X 7 6.7, 8,9 29 

STUDENT BACKGROUND 

Native Language Number of Students Countries of Origin Number of Students 

Spanish 67 United Stales 

Puerto Rico 
Dominican Republic 
Honduras 
Other (Xjuntries 



36 
19 
7 
1 
4 



Mean Years of Education in Native Coun!r\': 2.2; in the United States: 7.8 
Percentage of Stud e nts Eligible for the Free Lunch Programs: 100.0 

Most of the students' families had been in the United States for more than three years and were at or 
below the poverty level. They did not speak English as their primary language. Many of the students were 
deficient in reading and writing skills in their primary language. There were many one-parent households. In 
homes where there were two parents, usually one was employed. Veiy few parents came to the school voluntarily. 
If they came at all, it was usually at the school's request. 

ADMISSION CRITERIA 

The School Based Support Team (S.B.S.T.) and the Committee on Special Education (C.S.E.) selected 
students. Criteria for selection included Language Assessment Battery (LAB) scores at or below the 40th 
percentile, certification for special education, a chronological age bet^^-cen 10.9 and 21 as of September 1990, and 
difficulty functioning in a monolingual setting. 

The C.S.E. and the S.B.S.T. recommended students categorized with intensive social and emotional 
needs (cateogrized as in need of Specialized Instructional Environments [S.I.E.] VII) to Project CARES. All 
members of the S.B.S.T. were fluent in the Spanish language. 
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PROGRAMMING 



Features 

Career Awareness Resources for Exceptional Siudenu (Project CARES) targeted two publk schools with 
large numbers of ^>ecial education students of limited English proficiency (LEP). It provided the students with 
instruction in English as a Second Language (RS.L,), Native Language Arts (N.UA.), and bilingual instruction in 
the content areas. The project ^nwred a joint venture in whfch students, parents, supervisors, on- jnunity 
residents, and local business entrepreneurs introduced students to the job market. 

Strengths and Limitations 

The program raised the self-esteem and confidence of the students. The training program equipped the 
staff lo work together to achieve a common goal and the students with instruction in how to ck> peer tutoring. 
The project director reported that the project staff needed extra training on guidelines and procedures for the 
bilingual program. Some of the lax-levy teachers were not ecperienc^d bilingual teachers; and the ate 
coordinators, whose experience was also limited, required trainbg. 

CONCLUSIONS AND RECOMMEN'.IATSONS 

Project CARES was fully implemented. It m'Jt its objectives for the content area subjects of science, 
mathematics, and social studies. It met one of three career awareness objectives and partially met its objective for 
attendance. It failed to meet the objective for E.S.L. and a second career awareness objective, and did not supply 
the data needed to evaluate a third career awareness chjective. 

The conclusions, based on the findings of this evaluation, lead to the following recommendation: 

• Select or develop an instrument to measure occupational aspirations. 

• Supply OREA with all data necessary to evaluate all objectives. 



• Fjcplore ways in which to improve performance in E.S.L. 



PR0GR4M ASSESSMENT 



STAFFING 

Title Vn Staff (Total 3 ) 

Project Director 

Site Bilingual Coordinator 

Site Bilingual Coordinator 



Degree Language Competencies 



M.A. 
MA. 
M.S. 



Spanish 
Spanish 
Spanish 



""eaching^mmuaicative 
Profidendes CTPfCP) * 



TP 
TP 
TP 



Other Staff Working With Project Students (Total 12) 

Degree Certification Unguape Comnetendes 



B.A. 2 Bil. Common Branches 1 
B.S. 4 Bil. Spec. Ed. 2 
M.S. 6 Common Branches 1 

Speech 1 

Special Ed. 4 

T.P.D. 3 



Spanish 12 



Teaching^mmunicative 
Profidendes CTTfCPY 

AH TP 



IMPLEMENTATION AND OUTCOMES (Objectives prefaced by •) 

Tbc project relied heavily on computer-assisted instruction in mathematics, reading, and writing. 
Tlie career educatbn component provided the students with hands-on experiences at job sites. 

English as a Second Language fE.S.L.) 

E.S.L. was offered at a beginning, intermediate, or advanced level fK'e times a week at both sites. 

The teachers taught drama technic? ues with the National Textbooks LiN'ing in English. They used Barrent 
Publications' Writing Power series for teaching E.S.L. Students used the Josien Instructional Computer Program, 
a program of instruction in English concentrating on listening, reading, and writing skills. 



More advanced students tutored their peers in English. The low teacher-student ratio (ten students in a 
dass with a teacher and a paraprofcssional) facilitated whole-class and individual instruction. 



Teaching Proficiency (TP): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 
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• By June I'^l, 70 percent of the participating LEP students will show statistically s^ificant gains in 
English language proficiency, 

Evaluatktn Instrument: Language Assessment Battery (LAB)* 

Pretest: March 1990; posttest: March 1991. 
Number of students for whom pre- and posttest data were reported: 14 
Mean gain: 0.4 N.C.E.S (s.d.= 3.9) 
Mean gain is not statistically significant (t- 0.41, £>.05). 

Project did not meet objective. 

Native Language Arts fN.L.A.^ 

Estimated Percentage of Students Lacking Literacy Skills in Native Language: 85 

The teachers incorporated N.LA. in the E.S.L. component when students required it. 
The project did not propose an objectK'e for N.L A. 

Content Area Subjects 

Teachers taught content areas using E.S.L. methodology supplemented by native language instruction. 
The bilingual paraprofessionals assisted in providing instruction. 

They used the National Textbook Company's Nice and Easy grammar and Spanish for Beginners for 
instruction. They integrated N.L.A. activities with the content areas, using the Voluntad Publishers' The Names 
of Puerto Rico and Las Gotas dc Apua v Otras Poemas. Students engaged in peer tutoring. 

Teachers taught content areas using E.S.L. methodology supplemented by native language instruction. 

Science, Classes used the Addison Wesley's Science 4 and 5 textbooks, and Macniillan's The Earth and 
Its People. Teachers used experiments to promote hands-on activities. 

• By June 1991, 70 percent of the students enrolled in science will score at or above the passing 
criterion of 65 percent. 

Evaluation Indicator: Final course grades 

Number of students enrolled in a science course: 38 

Final grades at or above 65: 100 percent 

Project met objective. 



*OREA used a gap reduction evaluation design to evaluate the effect of supplementary instruction on projcci 
students' performance on the LAB. Since aU LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, and OREA used instead the group on which the LAB was normed. 
Test scores are reported in Normal Cuive Equivalents (N.C.E.s), which arc normalized standard scores with a mean 
of 50 and a standard deviation of 21.06. It is assumed that thf- norm group has a zero gain in N.C.E.s. in the absence 
of supplementary instruction and that participating students' gains arc attributable to project services. 



Mathematics . Teachers used the McGraw Hill texts, Practice Probtem SoMhe as well as computer^ 
assisted instruction. The Josten Instniciional Con^uter Program gave the i^uitents immediate reinforcement, as il 
verified answers to problen^ and corrected errors. Students also had bands^n experiences as they wrked at 
community pbs during the regular school day. 

• By June 1991. 70 percent of the students enrolled in mathematks will score at or above the passing 
criterion of 65. 

Evaluation Indicator: Final course grades 

Number of students enrolled in a mathematics course: 67 

Final grades at or above 65: 77.6 percent 

Project met objective. 

Social Studies . Teachers used the Barrent PubUcations' Cloze in Social Studies and Macmillan*s Earth 
and Its People . 

• By June 1991, 70 percent of the students enrolled in social studies will score at or above the passing 
criterion of 65. 

Evaluation Indicator: Final course grades 

Number of students enrolled in a social studies course: 67 

Final grades at or above 65: 79.1 percent 

Project met objective. 

Attendance 

• By June 1991, as a result of participating in the program, project students attendance will be 
significantly higher than that of mainstream students. 

To promote improved attendance, the project staff encouraged students to participate In the planning of 
their programs, contacted the parents of absent students, and met at least once a week uith students who were 
absent frequently. 

School Proicct Students' Rate Mainstream Students' Rate 

P.S. 12 63.2 51.7 

P. S. 754 75.0 71.0 

The difference between project and mainstrean: students' attendance rates for P.S. 12 was statistically 
significant (z= 1.74, 2=,05). 

The difference between project and mainstream students' attendance rates for P.S. 754 was not 
statistically significant (z- 0.47, |2>.05). 

Project met objective for P.S 12 but not for P-S. 754. 



RETENTION AND WITHDRAWAL 

The project did not propose ao objeciive in this area. 

Number of Students Percentage 

Retained in grade: 26 38.8 

Withdrew from program: 12 17.9 

Reasons for Withdrawal 
Transferred to other schools or programs: 6 (9.0 perccr*) 
Withdrawn from the program by parents: 3 (4.5 percent) 
Left the United States: 1 (1.5 percent) 

Left program for unknown reason: 1 (1.5 percent) 

Career Awareness 

Career education activities involved site visits to companies such as the Young Adult Institute, and visits 
b>- successful career people to talk about job requirements in their fields of expertise. The students also 
developed occupational scrapbooks and wrote articles for class newsletters telling about job opportunities. 

• By June 1991, as a result of participating in the career awareness/exploration class, 70 percent of 
enrolled students will show a significant improvement in occupational aspiration. 

Evaluation Instrument: Maier Occupational Aspiration Scale. 

The Maier Occupational Aspiration Scale does not exist. The project did not substitute another test, 
OREA was unable to evaluate objective. 

• By the end of the first project year, June 1991, the career resource spedalist hod curriculum 
specialists will have developed four instructional units in each of the content areas utilizing the 
learning strategics approach and career infusion technologies. 

Evaluation Indicator: Project-developed insf'jctional materials inveutor)' 

The career resource specialist and curriculum specialist did not develop the instructional units. 
Project did not meet objeciive. 

• By June 1991, all participating students will meet the bilingual career/resource specialUt at least once 
monthly. 

Evaluator Indicator: Program records. 

Students met with the appropriate staff members as proposed. 
Project met objeciive. 
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RFTFNTION AND WITIIDRAWAL 



N umber of Students 



Percentage 



Withdrew from prosram: 



Retained in grade: 



26 
12 



38.8 
17,9 



Reasons for Withdrawal 



Transferred to other schools or programs; 6 

Withdrawn from the program by parents: 3 

Left the United States: 1 

Left program for unknown reason: 1 



(9.0 percent) 
(4.5 percent) 
(1.5 percent) 
(1 J percent) 



MAINSTREAMING 

One student (1.5 percent of partidpants) was mainstreamed. 



CASE HISTORY 

G. was a nine yc.^rold female student who was bom in Santo Domingo and recently came to New York 
City. After being icsicd, she was placed in a fifth grade special education bilingual cla«. 

G. did well on the oral language test in Spanish and in picture vocabulary, but had difficulty with parts of 
speech. Although she understood both languages, she preferred speaking English. G.'s reading skills were at the 
first grade level She had problems with word attack skills and reading comprehension and lacked reading skills 
in Spanish. She was able to soh'e problems dealing with money and time but bad difficulty with measurement and 
fractions. G.'s attendance was poor. 

After some time in the project, G. showed a marked improvement in her attendance. The mvoWement of 
staff and rdated service providers has helped her improve her behavior. She was able to control her anger and 
use of profanity. Her mother has been extremely cooperative and has come to school any time she has been 
asked to do so. 

Although there has been improvement in G.'s behavior, it still keeps her from performing at a level 
consistent with her grade assignment. 
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APPENDIX A 



DATA CGLLECnON 
Student Data Fonn 

This is a one-page form completed by project staff on each participating and formerly participating (now 
mainsireamed) student. From the student data form OREA gathers data on demographics, academic outcomes, 
attendance, referrals, and student exit from the program. 



Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifwations, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview includes questions on program, student, or staff characteristics not answered on the 
questionnaire. It also allows project staff to offer qualitative data and to clarify points they have aUeady 
furnished. 



C itywide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.CE.s). N.C.E.S are 
normalized standard scores with a mean of 50 and a standard deviation of 2L06. They constitute an equal- 
intervai scale in which the distance is the same between any two adjacent scores. A gain of 5 N.CE.s is the sam 
whether it is at the lower or the higher end of the scale. N.CE.s can be used in arithmetic computations to 
indicate group progress. While percentile ranks are more familiar to many, percentile scales are not equal- 
interval :nd are unsuitable for such computations. 



Likert-Ts'pe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to Ove, are used in 
variety of ways. They examine attitudes in students toward school and career, native language use, and native 
mainstream cultures. Staff surveys and those for parents allow partidpants to evaluate workshops and staff 
development activities. 



H ATA ANALYSIS 

OREA , ses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project ojmparison group is avaUable in New York City, where all students of limited English proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norming group would show a gam of zero 
N.CE.s. 

To test whether pretest/posttest gains are greater than could be expected by chance alone, OREA uses a 
t-tcst To test whether a difference between two proportions (e.g., program and mainstream attendance rates) is 
greater than could be expected by chance, OREA uses a z-test. The level of significance is set at .05 for aU tests. 
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APPENDIX A 



DATA COLLECTION AND ANALYSIS 

Student Data Fonn 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionuaire indud.-s questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project r irector's Questionnaire. The interview also allows pi-oject staff to offer qualitative data or amplify 
responses to the Questionnaire. 

CiiN'wide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test sa)res are reported in Normal Curve Equivalents (N.C.E.s). N.CE.s are 
normalized standard scores with a mean of 50 and a standard deviation (s.d.) of 21.06. They constitute an equal- 
inten'al scale in which the distance is the same between any two adjacent scores. A gain of 5 N.C.E.s is the same 
whether it is at the lower or the higher end of the scale. N.C.E.s can be used in arithmetic confutations to 
indicate group progress. (Percentile scales, although more famiUar to many, are unsuitable for such computations 
since they are not equal-interval.) 

Likcrt-Tvpe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and nalKe 
and mainstream cuUures). They also assess staff and parent attitude and reactions to workshops and other 
activities. 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where aU students of limited English proficiency 
(LEP) are entitled to receive supplementaiy servit es, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students ujuld be attributed to project services, 
(See "Citywide Test Scares" above.) 

To test whether pre^^stiest gains are greater than could be expected by chance alone, OREA uses a 
t-lest. To determine whether a difference between two proportions (e.g., prograoi and mainstream attendance 
rates) is greater than could be expected by chance, OREA uses a z-test. The level of significance is set at .05 for 
all tests. 
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